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ABSTRACT 

Three 4-week institutes were held, one each in Los 
Angeles, St. Louis, and Philadelphia, to prepare competent school 
personnel at the state, county, and district levels to formulate, 
conduct, and evaluate educational experiments. Participants included 
teachers, principals, curriculum specialists, research consultants, 
and superintendents. The institute curriculum covered four broad 
areas: 1) the research proposal and report, 2) curriculum research 
and development, 3) curriculum evaluation paradigms and procedures, 
and 4) techniques for analyzing and interpreting experimental data. 
Training proceeded primarily by means of a series of instructional 
packages developed by the Southwest Regional Laboratory as part of 
their Staff Development Compendium. Followup assistance was also 
provided to participants. Gains were found between pre- and posttests 
of participants 1 knowledge of research methods, and a followup 
questionnaire showed that 25 percent of the participants had 
completed and obtained funding for research proposals. (Appendixes 
contain an outline of participant characteristics, a sample schedule 
of activities, and responses to the followup questionnaire.) (RT) 
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I. Significance of the Institutes to Education 



The purpose of the three four-week Institutes was to prepare compe- 
tent school personnel at the state, county, and district levels to for- 
mulate, conduct, and evaluate educational experiments. 

Although most educators are willing to acknowledge the desirability 
of controller experimentation In schools, such experimentation Is not 
now and never has been a conoon school enterprise. Experimental research 
Is widely thought to require a level of statistical and mathematical 
sophistication beyond that of the typical educator and to demand spe- 
cialised training In research methodology and techniques outside the 

1 Participants' stipends and allowances for the three 1967 Institutes 
were supported by a grant from the United States Office of Education, 
ESEA of 1965, Title IV, Grant No. OEG-1-7-O70674-3531. The institutes 
provided an opportunity for the tryout and evaluation of selected com- 
ponents of the Staff Development Cos?endiuia produced under the Staff 
Training project at the Southwest Regional Laboratory for Educational 
Research and Development. 

2 Editorial Insertions made to original document on 10-27-70 
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scope of the typical educational leader's normal training. Training 
programs for educators, even at the advanced graduate level, convert 
this thought into an operational obstacle. Thus, it is almost impossible 
to develop research competence in key school personnel within the frame- 
work of existing training programs. 

Universities Justifiably choose to invest In long-term programs of 
graduate study. Methodological research training typically follows a 
long and laborious route In which an attempt Is made to render the stu- 
dent a specialist In many fields, Including such areas as mathematical 
statistics, philosophy of science, computer design, and psychometric 
theory. While these areas Indeed have relevance to experimentation on 
Instruction, It Is unrealistic to expect people to be both willing and 
able to complete this kind of program and at the same time prepare them- 
selves to obtain and hold a key school position. Short of this, the 
only training typically available consists of research methods courses 
which Include discussion of experimentation as one part of the course 
but do not pretend to develop experimenters. 

One obvious solution Is for school systems to wait for and recruit 
the bright, young, long-term graduate student product of the degree 
program. Unfortunately, Instead of being attracted to a school system 
research position, the well-prepared holder of a research-oriented Ed.D. 

Is likely to be lured Into one of the Increasingly large number of col- 
lege and university staff vacancies. 

One might suggest that an easy solution would be to hire the compe- 
tent methodologist as a consultant to work In a cooperative fashion with 
school people. This Is not an optimal arrangement. In several situations 
where this had been tried, the activities of th two types of personnel 
closely approximate the parallel play activities of preschoolers; meth- 
odologists and school people do manage to work In the same general area 
but on separate aspects of the problem which are seldom Integrated Into 
common action. This Is not to say that cooperative activities are not 
possible and desirable. However, effective cooperation Is contingent 
upon school people learning more about methodology and methodologists 
learning more about the substantive situation. 

The three Institutes were designed to prepare key school personnel 
to formulate, perform, and/or monitor reapectaole Instructional research 
under natural school conditions. The participants were Involved In a 
concentrated program scheduled over a four-week period. The duration of 
the Institutes was carefully determined to accommodate participants with 
heavy job responsibilities. Key school people con arrange to be absent 
for a period of a month without disrupting the continuity of their own 
programs, but they simply cannot afford the conventional summer- school- 
length program. 




Greater concentration of program over a shorter period of time did 
not mean, however, that the institute curriculum crammed 15 courses into 
a single four-week time block. It was not Intended that the participants 
completing the Institute would be competent statisticians, psychologists , 
engineers, or even curriculum specialists. But if they achieved the 
objectives of the Institute, they should be able lo formulate, design, 
propose, conduct, analyze, and report instructional research that will 
pass the Inspection of specialists in each of the areas. There is no 
question that the participants were still novices in performing experi- 
ments; yet most had at least one proposal ready to begin work on by the 
end of the session. Moreover, as reported in the follow-up study 
(Appendix D) , their experiences in the institute predisposed continued 
research performance by most of them and Insured increased competency 
with further experience. 

An important aspect of the training program was the provision of 
opportunities for maintaining and extending the skills acquired in the 
program. Following the conclusion of the research-training institutes, 
at least two of the regional laboratories conducting them provided con- 
tinued training and feedback for the participants through a set of 
follow-up activities designed to assist in formulating and Implementing 
research proposals developed after the institutes. 

The Institutes gave the participant direct experience in using 
powerful concepts and procedures basic to school experimentation. At 
the same time, the participant was able to acquire sufficient sophisti- 
cation to recognize when he was in need ol specialized help. At that 
point he should be prepared to efficiently cbtaln the specialized help 
necessary. Many of the participants who attended have such help readily 
available either in the person of skilled personnel from the regional 
laboratories or specialists assigned to their office. 

I I . O bjectives of the Institute 

Emphasis during the institutes was placed upon assessing the 
adequacy of the SWRL-developed materials to provide instructional con- 
tinuity and yield dramatic gains in researcher performance in the several 
instructional contexts not directly controlled by Laboratory personnel. 
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Following are the Instructional objectives common to all Institute 
locations and related to the SWRL Sta ff Development Compendium sequences. 

1. Describe operationally the conditions to be met In completing 
a written proposal for an educational experiment. 

2. Distinguish between educational experiments and other forms 
of professional education endeavors. 

3. Define: the expected outcomes of learning and Instruction In 
terms of observable learner behavior amendable to empirical 
Investigation. 

4. Describe In operational terms the planned classroom trans- 
actions for a given curricular program. 

5. Specify appropriate Independent and dependent variables lor 
an experimental study, and state the specific relationships 
to be Investigated In the study. 

6. Select valid and practical experimental designs for Investi- 
gating specified relationships. 

7. Construct and/or select valid criterion instruments to measure 
the extent to which the desired outcomes of a program have 
been attained. 

8. Perform and/or monitor the experimental operations consistent 
with the design selected, 

9. Select the most appropriate statistical procedures for ana- 
lysing the experimental data. 

10. Perform and/or monitor the necessary statistical analyses 
using the most appropriate computing vehicle (emphasis placed 
on electronic data processing and practice with the accompa- 
nying user manual). 

11. Interpret the results of an experiment, with special emphasis 
on the Implications for curriculum decisions and further 
experimentation, 

12. Describe the characteristics of b report which presents the 
results of an experiment In forms suitable for dissemination 
to various audiences. 
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t X I . Number of Trainees and Selection Criteria 



Trainees wen; selected by each of the cosponsoring Institutions. 
Participants vrere invited to attend the institute serving their particular 
geographic are.*, Criteria for geographic and Institutional representation 
were left to the discretion of the cosponsoring agencies. Paragraph 2, 
Appendix A shows the geographic distribution of the participants In the 
three locations. Participants were drawn almost exclusively from the 
regions represented by the respective laboratories. 

The general purpose of the Institutes necessitated selection cri- 
teria which emphasized sampling on a geographic basis from school person- 
nel presently Involved In educational leadership positions. Although it 
was hoped that the general mental ability level would be high, primary 
consideration was given to the type of position held and the geographic 
area served. 

Priority applicants, In general, held administrative posts In state 
departments of public Instruction, county offices of Instruction, or In 
public or private elementary or secondary school districts. Final 
selection depended upon the specific nature of their position for the 
coming year. They were to have major responsibilities such as the for- 
mulation and development of federally assisted programs, the operational 
direction and evaluation of such programs, or the direction of curricu- 
lum research and development and/or Instruction. 

However, the three locations ended up dlf feeing somewhat in their 
selection criteria. Los Angeles selected administrative personnel who 
were functioning quite directly in curriculum. Phii elphla, on the 
other hand, chose staff level personnel who were to perform as special- 
ists of one kind or another. Too, alnvst o.ie-quarter of their group 
came from the classroom teacher ranks as contrasted with zero In 
St. Louis and three percent In Los Angeles. The St. Louis group was 
comprised of high level administrative and stAff personnel; sixty per- 
cent were either superintendents or principals. Although all partlcl* 
pants selected had direct responsibility for curriculum research and 
development, they differed from location to location In terms of 
remoteness from the dally operations Involved. Appendix A contains 
data that further describe the several groups of participants. 

Within this context additional consideration was given to possession 
of the following characteristics) 

1. general mental ability competitive with the best of the 
applicants as measured by the Miller Analogies Test . 

2. evidence of a maximum number of years of future continued 
service to professional education judged relative to the none 
of the applicants, Final participant selection was the 
responsibility of each of the regional selection connittees. 
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IV. Program and Materials 

The Southwest Regional laboratory for Educational Research and 
Development (SWRL) has formally recognized a responsibility for direct 
involvement in the preparation of educational research anc evaluation 
training materials and procedures. The Staff Training activity is com- 
mitted to the design and construction of generallzable Instructional 
sequences that can be used by a wide variety of educational personnel. 

I nstructional Sequenc es 

As Indicated a numbor of "self-contained" instructional sequences, 
in various stages of formulation and development in the Laboratory, were 
used. Much of the material which forms the nucleus of the prototype 
sequences has been tried out by SWRL staff during previous Title IV 
Research Training Institutes and at workshops on curriculum evaluation 
held in Minneapolis, Washington, D.C., Santa Barbara, Riverside, 

Las Vegas, San Diego, and elsewhere. 

The Sta ff Development Compendium is comprised of two categories of 
sequences. The Curriculum Evaluation sequences were designed to develop 
the skills necessary for the educational leader to make empirically 
grounded decisions concerning revision and development of curriculum 
materials . 

The Instructional Research sequences were designed to provide 
educational personnel with the necessary methodological skills to initiate, 
conduct, and report experiments in school learning and instruction. 
Development of these sequences was based on the assumption that controlled 
experimentation on school learning and instruction is critical and that 
controlled experiments should actually be conducted in the schools. 

The sequences were labeled "self-contained" to reflect that a mini- 
mum of staff monitoring should be necessary for instructional success. 

One of the major objectives was to develop materials and procedures that 
would be maximally effective under a variety of instructional conditions. 
The history of this program area suggests that a modular approach to tho 
development of Instructional materials and procedures provides more 
options And permits greater freedom of choice to the instructor, At the 
same time, the nature of the components helps to Insure that continuity 
is not ignored and student performance is monitored. 

Below is a list of the prototype instructional sequences developed 
in the Laboratory and tried out in this series of institutes. 

Prototype Sequences 

1. Describing Educational Outcomes 

2. Constructing Behavioral Objectives 

3. Classifying Educational Research Studies 
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4. Interpreting Research 

5. Improved Educational Programs 

6. Types of Instructional Materials 

7. Defining Instructional Specifications 

8. Management of Behavioral Consequences 

9. Measurement of Educational Outcomes 

10. Absolute vs Relative Criteria 

11. Construction of Prototype Items 

12. Educational Criterion Measures 

13. Threats to the Validity of a Study 

14. Minimizing Threats to the Validity of a Study 

15. Design Paradigms and Procedurer 

16. Use of Library Computer Programs 

17. Choosing an Appropriate Statistical Procedure 

18. Analyzing Variability 

19. A Factorial Model: Rules of Thumb for the ANOVA 

20. Formulating the Research Proposal 

21. Components of the Research Proposal 

22. The Research Report 

Management of the Institutes 

Since the above sequences have been used previously only in small 
and/or short tryouts, this series of institutes constituted the first 
large-scale quality verification cycle. Therefore, the following staff- 
ing procedures were followed: 

1. One member of the SWRL professional staff was responsible for 
coordinating all three lnatltutes. 

2. One staff member from each oi the three regional laboratories 
represented waa responsible for coordinating the respective 
area institutes, 

3. Each of the cooperating regional laboratories supplied the 
additional staff necessary to monitor and augment the 
Instructional sequences. 

4. SWRL provided the necessary professional cadre to accompany 
certain of the "critical." materials. That Is, for those 
objectives for which sequences had not been completed, or 
where additional instruction and/or discussion was necessary, 
SWRL staff was on-site during Instruction. 
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5, In late April 1967, draft copies of the relevant (sequences. 
Including user manuals, were delivered to the personnel In the 
other laboratories who were responsible for Instruction at 
that site. 

6. A three-day training session was held in Tempe, Arizona, to 
Insure maximum effectiveness of "package delivery," and to 
rehearse the roles of the instructors. Following Is the sched- 
ule followed during the session. 

Tuesday, Hay 23 
ORIENTATION 

Introduction of Sumner Institute Staff 
Introduction of Staff Training Personnel 
Institute Dates, Location, Personnel 
Rationale for SWRL Involvement 
Condition of Cooperative Laboratory Involvement 
Summer Institute Proposal 

ADMINISTRATIVE CONSIDERATIONS 
Fiscal Procedures 
Personnel Responsibilities 

Physical Facilities, Equipment and Supply Require- 
ments 

Institute Libraries 
Pre-Institute Orientation Packet 

Wednesday, May 24 

CURRICULUM CONSIDERATIONS 
Objectives 

Instructional Sequence Breakdown 
Tentative Curriculum Schedule 
INSTRUCTIONAL USB OF THE SEQUENCES 
Self-Contained Sequences 
User's Manual 

SEQUENCE DESCRIPTIONS AND DISCUSSION 
INDIVIDUAL LABORATORY PLANNING 
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Thursday. May 25 

PACKAGE DESCRIPTION AND DISCUSSION 
EVALUATION CONSIDERATIONS 
Participant Data 
Evaluation Instruments 
Staff Evaluation 

Monitoring and Feedback Procedures 
Follow-Up Procedures 
Fiscal Accounting 
Final Report 



V. Description of Activities 



Schedule 



The Institutes had scheduled activities from 8 vo Hi 30 a.m. and 1 
to 4 p.m. Monday through Friday. In general, the rooming sessions were 
devoted to full-group work on the current sequences and related lecture- 
demonstration-discussion. The afternoon sessions were usually devoted 
to small group consultation with Individual staff members, completing 
assigned criterion exercises to demonstrate mastery of the sequence 
objectives contldered In the morning session, and individual study. It 
was hoped that by building "free’ : Individual study Into the dally scheo 
ule the pacing problems associated with self-contained packages would be 
minimized. 

One can rather artlflcally divide the objectives and content Into 
four conventional categories (numbers in parenthesis refer to the objec- 
tives listed in Section II): 

1. The research proposal and report. (1, 12) 

2. Curriculum research and development. (2, 3, 4, 5) 

3. Curriculum evaluation paradigms and procedures. (6, 7, 8) 

4. Techniques for analysing and interpreting experimental data. 

(9, 10, 11) 

To capitalise on the Interrelatedness of the content and objectives, the 
materials were Introduced more or less according to the order shown in 
pnges 6 and 7. To provide for the needed flexibility in programming, the 
actual schedule was formalised each Friday for the subsequent week. 

Sanple institute schedules are shown in Appendix B» 
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University Course Credit 



With the exception of California, arrangements were made by the 
cooperating regional laboratories to jointly sponsor their respective 
Institutes with a local college or university. Participants were 
optionally permitted to enroll In the graduate college for four semester 
hours of graduate credit. This was block credit, rather than credit for 
separate aspects of the Institutes. The Institute In California was 
cosponsored by the Los Angeles City School System, thus no credit was 
opted for. 

Description of the Sequences 

The institutes were concerned with talkiug about research and 
experimentation, but with developing competency to do It. The materials 
were designed and arranged so that the participant was able to monitor 
his own performance. ' Each sequence contained expository material that 
directed his progress. Some sequences contained all of the Instructional 
material, while others from time to time directed the student to read an 
article or book not found in the sequence. In any case, common to all 
packages was a series of carefully sequenced exercises designed to cover 
each aspect of the relevant objective being considered. Every effort 
was made to minimize the "busywork" and the mechanical aspects of each 
exercise, and at the same time insure their relevance to the natural 
school situation. 

Mastery Exercises 

In addition to the enroute series of exercises described above, 
an "off-line" exercise covering the objectives t» w t by each sequence was 
administered to the student. These were reasonably psychometrtcally 
adequate criterion Instruments used to measure the Intended outcomes. 

Too, they were used to tie performance back Into the instructional trans- 
actions and antecedents for revision of materials. 

Small Group Consultation 

Although handled differently at each location, It was anticipated 
that there would be some communality of interest and problems among the 
participants which could be handled most efficiently via small group 
sessions, aided by individual staff members. The groups were to be 
"open-ended" and highly fluid. The emphasis was to be placed on problem 
solving rather than group dynamics. As quickly as relevant aspects of 
the situation were, explored and exhausted, a given group was to be dis- 
solved and the participants would go on to something else. Although a 
certain amount of this actually took place, the bulk of this time was 
directed to specific objective-related lectures and discussions. 
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Individualized Study 

Participants were encouraged to pursue aspects of the institute in 
which they become individually interested* While it was clearly stated 
that the institute objectives were to be achieved by each participant, 
the staff was to make every effort to stimulate additional efforts* In 
the past our institute participants have been sufficiently motivated to 
engage in this kind of activity during their off hours* 

Follo w-Up Activities 

Two types of follow-up activities were planned. First, the 
cooperating regional laboratories were to establish and maintain a 
research monitoring service for institute participants. This service 
enabled participants to utilize certain of the available laboratory 
technical resources in designing current and future research projects 
and in analyzing evaluation data. Research for Better Schools utilized 
this type of strategy* Second, the instructional materials developed 
for research-training use by the laboratories and all other relevant 
research and evaluation materials circulated within the laboratories 
would be routinely mailed to participants after the conclusion of the 
institutes* These activities should have enabled participants to main- 
tain and extend their research and evaluation skills on an individualized 
basis'. Unfortunately, only limited distribution was made due to lack of 
funds and the ambiguity surrounding copyright law interpretations. 

VI. Summary 

The thrt ‘ institutes successfully demonstrated the: 

1. efficacy of the modular, '’self-contained" approach to packag- 
ing instruction. 

2. flexibility of the modules to accommodate unique user interests 
and requirements. 

3. adaptability of the sequences to varied levels of user and 
instructor sophistication* 

4. utility of the sequences as a post-institute reference library. 

5. power of the sequences to promote continuous meaningful 
dialogue about practical problems while maintaining necessary 
instructional continuity. 

6. feasibility of managing instruction by objectives, assuming 
a defined instructional system. 

7. importance of carefully specifying and conducting instructor 
training prior to such institutes. 
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Participant Test Performance Results 



Items on the criterion referenced rests sampled heavily from 
"terminal objective" type R. & D. situtations. Thus performance gains 
made were toward the higher levels of difficulty in terms of application 
of the concepts and techniques to "natural" research settings. 

The objectives treated during .'■.he institutes constituted a formid- 
able list for participant mastery during a four-week period. However, 
the performance profiles for the three institutes illustrate dramatic 
gains for all four objectives clusters. Tables I, II, and III show pre 
and posttest results for each institute location. The scores are 
expressed as percentage of items correct for each cluster of objectives. 
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Table I 

*Group Profile for Los Angeles: Median Scores (percentage) for Pre and Posttest, by Objectives Cluster 

Percent Mastery 



C luster 


Content 


Percentage 
Pre Post 


1 


Research Proposal and Report (1, 12)** 


64 


73 


2 


Research and Development Strategies 
(2, 3, 4, 5) 


61 


74 


3 


Evaluation Paradigms/Procedures 
(6, 7, 8) 


48 


67 


4 


Analyzing and Interpreting Data 
(9, 10, 11) 


27 


64 
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Table II 

*Group Profile for Philadelphia: Median Scores (percentage) for Pre and Posttest, by Objectives Cluster 



Cluster 


Content 


Pre 


Post 


1 


Research Proposal and Report (1, 12)** 


52 


65 


2 


Research and Development Strategies 
(2, 3, 4, 5) 


55 


73 


3 


Evaluation Paradigms/Procedures 
(6, 7, 8) 


44 


68 


4 


Analyzing and Interpreting Data 
(9, 10, 11) 


35 


64 
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Table III 

*Group Profile for St. Louis: Median Scores (percentage) for Pre and Posttest, by Objectives Cluster 



Cluster 


Content 


Pre 


Post 


1 


Research Proposal and Report (1, 12)** 


54 


67 


2 


Research and Development Strategies 
(2, 3, 4, 5) 


59 


69 


3 


Evaluating Paradigtns/Procedures 

<6, 7, 8) 


49 


67 


4 


Analyzing and Interpreting Data 
(9, 10, 11) 


33 


60 



Pet cent Mas frery 



I 
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* dotted line relates to pretest results; solid relate to posttest results 
** numbers in ( ) refer to the objectives described under section II of the report 
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In addition to the posttest, participants were asked to respond to 
a series of seven-point scales made up of six bi-polar continua for each 
instructional sequence. Appendix C shows the combined results for each 
instructional sequence, expressed in terms of the percentage of total 
responses. The instructional sequences were quite favorably received. 

Albeit, many of the items on the criterion tests proved very 
difficult, requiring extension of acquired skills to n out-of-institute n 
research settings. However, the tests were designed to assess the full 
range of "criterion" performance without regard for instructional con- 
straints. Trade-offs always have to be made in such situations, If 
the instructional goat (in say a four-week workshop) is to maximize 
performance on criterion referenced tests, the n one must structure the 
environment giving full attention to three obvious problem areas: 

, heterogeneity of participants 

- level of entry skills 

- areas of interest 

- job requirements 

, artificiality of an academic setting 

. possible mismatch between outcomes, performance standards, 

and the reality of the time frame. 

These institutes suggest that effective management of instruction 
by objectives is possible under a fairly tolerant range of if - then 
statements. The critical aspect is to clearly specify the "if" side 
of the statements so that an adequate analysis of consequences for the 
"then" side may be completed and translated into instruction. 

For example, jlf the three instututes singular goal was to maximize 
achievement of the 12 aformentioned Compendium objectives, as measured 
by the criterion referenced tests, the n, the instructional conditions 

homogenize the participants in terms of entry skills, interests, 
and job expectations, 

provide a monitoring strategy for the control of enroute mastery 
of concepts and skills. 

develop a mechanism within the institute setting for applying 
skills and concepts to real-time problems familiar to the 
participants , 

provide techniques for monitoring the transfer of acquired 
skills to participants 1 own research setting. 



must: 
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If, on the other hand, there are other objectives, then they must 
be specified and their consequences for instruction analyzed. Those 
who plan to engage in future training activities should be aware of the 
significance of the above, particularly if they are going to be held 
accountable for specified outcomes. 

Follow-up 

On December 4, 1957, a follow-up questionnaire was mailed to the 
participants of all three institutes. Appendix D contains the summary 
of the data collected. Of the 86 mailed, 68 were completed and returned 
in time for analysis. Highlights of the returns are: 

1. Thirty-seven percent of the participants are presently (1967-68) 
engaged in greater than half-time research and evaluation 
activities, as opposed to only 14 percent during 1966-67. 

2. Over one-quarter of the 1967-68 activities were related to 
curriculum evaluation. 

3. Twenty-five percent of the responding participants actually 
completed the proposal they initiated during the institute and 
have received financial support for the conduct of the activity. 

4. When asked to indicate those institute materials to which they 
have referred frequently subsequent to the institutes, the 
following resulted: 



a. 


behavioral objectives 


52 percent 


b. 


research proposal 


34 percent 


c. 


research report 


33 percent 



5. However, 64 percent of the participants reported that all of 
the materials have assisted them ''very much" with their post- 
institute activities. 

6. Twenty- two percent of the respondents reported that they have 
changed positions since the institute; almost all of them 
reported that attending the institute was instrumental to the 
change . 
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VI. Participating Staff 

A. University of Pennsylvania (July 10-August 4, 1967) 

Robert L. Baker, Southwest Regional Laboratory 
Gerald Chalmers, Research for Better Schools 
Robert Fried, University of Pennsylvania 
Thor Krogh, Research for 3etter Schools 
Fred Pyrczak, Research for Better Schools 

B. Llndenwood College, Missouri (June 26-July 22, 1967) 

Robert Berger, Southwest Regional Laboratory 

Edwin Bridges, Washington University 

Robert Elsea, Washington University 

Earl Morris, Central Midwestern Regional Educational 
Laboratory 

Howard Russell, Central Midwestern Regional Educational 
Laboratory 

Donald Thompsen, Central Midwestern Regional Educational 
Laboratory 

C. Inglewood, California (July 5-August 1, 1967) 

Larry Harty, Southwest Regional Laboratory 
Bruce Monroe, Southwest Regional Laboratory 
Carolyn Owen, Southwest; Regional Laboratory 
Carolyn Wilkerson, Southwest Regional Laboratory 
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APPENDIX A 



Educational Innovation: Research and Evaluation Techniques 



Description of the Participants 



1. How many applications 
were received? 



Received 



Los Angeles 


97 


Philadelphia 


100 


St. Louis 


60 




257 



Qualified 


Accepted 


Completed 


65 


31 


31 


47 


29 


29 


52 


26 


25 


164 


86 


85 



2. What geographical areas 
were represented? 

Los Angeles 
Philadelphia 
St, Louis 



3 states and Washington, D.C. 
7 states 

3 stat es 

15 states and Washington, D.C. 



3. How old were the 
participants? 



Average age Range 



. os 


Angeies 


38 


26-64 


PL 


1 adelphi.a 


40 


23-61 


St. 


Loui s 


43 


31-62 



4. Were both sexes 
represented? 





Male 




Female 


%_ 


Los Angeles 


25 


81 


6 


19 


Philadelphia 


20 


69 


9 


31 


St. Louis 


23 


88 


3 


12 




68 


79 


18 


21 
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5. What educational leve’ 
did they represent? 



Los Angeles 
Philadelphia 
St. Louis 



BA MA Ed. S. Ph.D. 



4 16 3 

6 21 0 

_2 24 0 

12 61 3 



8 

2 

_0 

10 



6. What were the Miller 
Analogies Test raw 
scores? 





Mean* 


Range 


Los Angeles 


54 


29-90 


Philadelphia 


62 


32-92 


St. Louis 


48 


23-88 



7. What were present primary 
job responsibilities? 



L.A. 



Phil. 



St, Louis No. 7„ 



a. Principal 6 

b. Research associate/ 9 

consultant 

c. Curriculum supervisor 2 

d. Director of research/ 4 

evaluation 

e. Superintendent 2 

f. Director of curriculum 4 

g. Classroom teacher 2 

h. Director of personnel 1 

i. Assistant superintendent 1 

j. College professor 0 

k. Librarian _0 

31 



1 

3 

9 

2 

0 

2 

6 

2 

1 

2 

JL 

29 



9 16 19 

1 13 15 

1 12 14 

5 11 13 



7 

3 

0 

0 

0 

0 

_0 

26 



9 10 

9 10 

8 9 

3 3 

3 3 

2 2 

_1 1 

86 100 




*A raw score of 53 is at the fiftieth percentile for entering 
graduate students in educational administration. 
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8. What percentage of time do they spend In research and evaluation 
activities? 



Percent- 




ELi.L. 


1966-67 

St.L. 


Total 
Mj. 2L_ 


L.A. 


Phil. 


1967-68* 

St.L. 


Total 

No*. °L_ 


age 

90-100 


2 


6 


0 


8 


9 


4 


10 


1 


15 


18 


80-89 


1 


0 


0 


1 


1 


1 


0 


0 


1 


1 


70-79 


0 


1 


0 


1 


1 


1 


1 


1 


3 


3 


60-69 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


50-59 


2 


2 


0 


4 


5 


9 


8 


6 


23 


27 


40-49 


0 


0 


0 


0 


0 


0 


0 


1 


1 


1 


30-39 


1 


0 


0 


1 


1 


2 


0 


1 


3 


3 


20-29 


5 


2 


2 


9 


10 


6 


3 


10 


19 


23 


10-19 


5 


4 


8 


17 


20 


6 


0 


4 


10 


11 


0-9 


15 


14 


16 


45 


52 


2 


77 


2 


11 


13 


















31 


29 


26 


86 


100 


31 


29 


26 


86 


100 



What are their primary research 


interests? 










Location 








L.A. 


Phil. 


St. Louis 


a. 


Instructional proce- 










dures 


12 


8 


8 


b. 


Federal project eval- 










uation 


8 


4 


2 


c. 


Administration 


5 


2 


6 


d. 


Reading 


4 


4 


5 


e. 


Attitudes /motivation 


2 


4 


1 


f. 


Teacher education 


0 


4 


0 


g. 


Compensatory education 


0 


0 


4 


h. 


Vocational education 


0 


3 


0 






31 


29 


26 



Total 
No. % 



28 32 

14 16 

13 15 

13 15 

7 8 

4 5 

4 5 

_3 4 

86 100 




*Frora Follow-Up 3tudy summarized in Appendix D. 
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APPENDIX C 

Percentage of Participants Responding to 
Each Category, by Rating Dimension and Instructional Sequence 





1. Describing Educational Outcomes 




useful 


36 27 


17 


15 


4 


3 


0 


useless 


confusing 


0 13 


15 


20 


13 


28 


10 


clear 


orderly 


24 30 


15 


13 


17 


0 


0 


chaotic 


incomplete 


0 7 


2 


20 


20 


30 


22 


complete 


convenient 


19 30 


16 


19 


9 


7 


0 


inconvenient 


complex 


0 24 


24 


30 


11 


7 


4 


simple 




2. Constructing Behavioral Objectives 




useful 


29 37 


21 


10 


0 


3 


0 


useless 


confusing 


0 3 


5 


5 


39 


27 


21 


clear 


orderly 


27 27 


34 


5 


8 


0 


0 


chaotic 


incomplete 


0 0 


8 


3 


33 


38 


16 


complete 


convenient 


16 43 


24 


3 


10 


3 


0 


inconvenient 


comp lex 


3 10 


13 


18 


31 


18 


5 


simple 




3. Classifying Educational Research 


Studies 




useful 


64 21 


14 


0 


0 


0 


0 


useless 


confusing 


4 7 


0 


7 


11 


29 


42 


clear 


orderly 


54 29 


7 


11 


0 


0 


0 


chaotic 


incomplete 


0 0 


0 


4 


19 


37 


41 


complete 


convenient 


42 29 


14 


7 


4 


0 


4 


inconvenient 


complex 


0 7 


21 


14 


18 


14 


25 


simple 
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4. Interpreting Research 



useful 


47 


42 


9 


2 


0 


0 


0 


useless 


confusing 


0 


1 


2 


8 


11 


37 


37 


clear 


orderly 


44 


40 


13 


1 


0 


1 


0 


chaotic 


Incomplete 


0 


2 


1 


13 


9 


55 


20 


complete 


convenient 


40 


35 


13 


2 


1 


2 


7 


Inconvenlen 


complex 


1 


6 


9 


36 


17 


28 


3 


simple 



5. Improved Educational Programs 





















useful 


24 


46 


14 


13 


1 


1 


1 


useless 


confusing 


0 


4 


3 


12 


17 


37 


27 


clear 


orderly 


33 


37 


18 


8 


1 


2 


1 


chaotic 


Incomplete 


11 


16 


6 


16 


15 


22 


15 


complete 


convenient 


28 


38 


11 


19 


2 


0 


1 


Inconvenient 


complex 


0 


30 


11 


19 


22 


28 


12 


simple 






6. Types of Instructional 


Materials 




useful 


28 


31 


20 


11 


8 


1 


0 


useless 


confusing 


1 


2 


4 


13 


14 


33 


33 


clear 


orderly 


30 


33 


20 


14 


2 


1 


0 


chaotic 


Incomplete 


0 


10 


4 


19 


19 


33 


16 


complete 


convenient 


34 


18 


23 


21 


4 


0 


0 


Inconvenient 


complex 


0 


10 


5 


31 


19 


18 


17 


simple 




7. 


Defining Instructional Specifications 




useful 


18 


22 


30 


20 


4 


6 


0 


useless 


confusing 


4 


2 


13 


25 


26 


11 


13 


clear 


orderly 


10 


47 


18 


16 


4 


4 


0 


chaotic 


incomplete 


2 


2 


8 


18 


29 


24 


16 


complete 


convenient 


17 


20 


20 


26 


4 


11 


2 


lnconvenle.it 


complex 


8 


14 


24 


22 


22 


6 


2 


simple 
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8. Management of Behavioral Consequences 



useful 


31 


31 


21 


10 


7 


2 


2 


useless 


co._ fusing 


3 


5 


3 


11 


15 


40 


23 


clear 


orderly 


25 


35 


15 


17 


2 


7 


0 


chaotic 


incomp le te 


21 


40 


11 


19 


2 


5 


2 


complete 


convenient 


2 


3 


5 


15 


22 


32 


22 


inconvenient 


complex 


5 


5 


11 


31 


21 


19 


8 


simple 




9. 


Measurement 


of Educational Outcomes 




useful 


33 


16 


21 


16 


13 


0 


0 


useless 


confusing 


4 


8 


4 


8 


21 


13 


42 


clear 


orderly 


33 


16 


33 


8 


4 


0 


4 


chaotic 


incomplete 


4 


0 


0 


8 


38 


33 


16 


complete 


convenient 


16 


38 


13 


16 


13 


4 


0 


inconvenient 


complex 


0 


0 


8 


21 


38 


16 


16 


simple 




10. Absolute 


vs Relative 


Criteria 




useful 


og 


26 


13 


13 


0 


13 


0 


useless 


confusing 




0 


0 


13 


0 


50 


38 


clear 


orderly 


38 


38 


26 


0 


0 


0 


0 


chaotic 


incomplete 


0 


0 


0 


26 


0 


63 


13 


complete 


convenient 


38 


13 


38 


0 


0 


13 


0 


inconvenient 


complex 


0 


0 


0 


0 


0 


38 


63 


simple 
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. Construction o 


f Prototype 


Items 




useful 


31 


31 


17 


8 


6 


8 


0 


useless 


confusing 


0 


3 


8 


16 


5 


32 


35 


clear 


orderly 


28 


28 


22 


17 


3 


3 


0 


chaotic 


Incomplete 


0 


0 


5 


19 


19 


33 


22 


complete 


convenient 


16 


39 


11 


16 


8 


8 


0 


inconvenient 


complex 


0 


3 


14 


25 


19 


U 


28 


simple 



O 




26 



12. Educational Criterion Measures 



useful 


32 


26 


25 


10 


4 


1 


1 


useless 


confusing 


3 


6 


12 


18 


28 


31 


18 


clear 


orderly 


24 


36 


20 


11 


7 


4 


0 


chaotic 


incomplete 


3 


3 


5 


17 


19 


34 


19 


comp lete 


convenient 


22 


30 


30 


10 


3 


4 


1 


inconvenient 


complex 


1 


8 


7 


32 


10 


30 


11 


simple 




13. 


Threats to 


the Validity of a 


Study 


useful 


58 


19 


11 


11 


0 


0 


0 


useless 


confusing 


4 


11 


7 


21 


14 


30 


13 


clear 


orderly 


25 


29 


20 


14 


10 


2 


0 


chaotic 


incomplete 


0 


2 


8 


13 


11 


38 


28 


complete 


convenient 


30 


26 


19 


8 


8 


8 


2 


inconvenient 


complex 


17 


17 


22 


23 


6 


7 


4 


simple 


14. 


Minimizing 


Threats to 


the Validity 


of a 


Study 


useful 


47 


25 


13 


9 


2 


4 


0 


useless 


confusing 


9 


8 


25 


30 


11 


8 


9 


dear 


orderly 


13 


23 


25 


21 


13 


4 


2 


chaotic 


incomplete 


0 


8 


10 


21 


10 


29 


23 


complete 


convenient 


10 


27 


25 


17 


8 


12 


2 


Inconvenient 


complex 


28 


21 


26 


13 


6 


0 


6 


simple 




15. 


Design Paradigms and Procedures 





useful 


50 


36 


;o 


5 


0 


0 


0 


useless 


confusing 


12 


15 


12 


15 


20 


17 


10 


clear 


orderly 


30 


38 


18 


8 


e 


3 


0 


chaotic 


incomplete 


5 


0 


2 


14 


5 


36 


38 


complete 


convenient 


22 


24 


20 


12 


10 


7 


5 


inconvenient 


complex 


29 


17 


19 


19 


iO 


5 


2 


simple 



ERIC 
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16. Use of Library Computer Programs 



useful 


62 


20 


15 


2 


0 


0 


2 


useless 


confusing 


0 


0 


0 


4 


12 


23 


33 


clear 


orderly 


25 


13 


4 


4 


8 


6 


40 


chaotic 


incomplete 


0 


4 


0 


6 


15 


42 


32 


complete 


convenient 


52 


25 


12 


4 


0 


6 


2 


Inconvenient 


complex 


0 


2 


8 


25 


12 


29 


25 


simp le 


17. 


Choosing an Appropriate 


Statistical 


Procedure 


useful 


30 


26 


15 


13 


13 


2 


0 


useless 


confusing 


15 


13 


17 


11 


15 


20 


9 


clear 


orderly 


13 


23 


15 


26 


6 


13 


4 


chaotic 


Incomplete 


7 


11 


9 


28 


13 


15 


17 


complete 


convenient 


14 


30 


9 


21 


2 


7 


16 


Inconvenient 


complex 


9 


2 


27 


41 


14 


5 


2 


simple 






18. 


Analysing Varlabll'ty 






useful 


35 


23 


20 


10 


5 


8 


0 


useless 


confusing 


28 


8 


18 


20 


18 


10 


0 


clear 


orderly 


8 


15 


23 


23 


5 


23 


5 


chaotic 


lncomp lete 


10 


2 


10 


30 


7 


22 


20 


complete 


convenient 


8 


16 


24 


19 


3 


11 


19 


Inconvenient 


complex 


31 


24 


22 


15 


5 


2 


0 


simple 


19. A 


Factorial Model: 


Rules 


of Thumb for 


the ANOVA 


useful 


11 


16 


16 


37 


11 


5 


5 


useless 


confusing 


20 


15 


25 


25 


5 


10 


0 


clear 


orderly 


21 


21 


5 


32 


11 


11 


0 


chaotic 


Incomplete 


5 


0 


5 


21 


0 


58 


11 


complete 


convenient 


5 


16 


5 


26 


5 


26 


16 


Inconvenient 


complex 


35 


35 


11 


20 


0 


0 


0 


simple 
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20. Formulating the Research Proposal 



useful 


41 


31 


24 


9 


4 


0 


0 


useless 


confusing 


0 


3 


7 


3 


19 


40 


28 


clear 


orderly 


23 


45 


15 


8 


8 


2 


0 


chaotic 


incomplete 


0 


0 


10 


22 


18 


35 


15 


complete 


convenient 


23 


42 


11 


15 


6 


4 


0 


Inconvenient 


complex 


2 


6 


13 


28 


17 


28 


7 


simple 


useful 


21. 

56 


Components of the 
20 18 4 


Research Proposal 
0 0 2 


useless 


confusing 


0 


6 


6 


6 


17 


36 


30 


clear 


orderly 


33 


39 


22 


4 


0 


2 


0 


chaotic 


incomplete 


2 


11 


7 


14 


18 


45 


2 


complete 


convenient 


33 


30 


24 


9 


0 


4 


0 


Inconvenient 


complex 


0 


0 


9 


48 


22 


13 


7 


simple 


useful 


53 


22. 

33 


Ihe Research Report 
6 5 2 0 


2 


useless 


confusing 


0 


3 


5 


10 


10 


38 


35 


clear 


orderly 


38 


31 


14 


11 


0 


3 


3 


chaotic 


Incomplete 


34 


30 


14 


17 


2 


2 


2 


complete 


convenient 


0 


2 


2 


13 


8 


34 


42 


Inconvenient 


complex 


3 


3 


9 


31 


19 


27 


8 


simple 
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29 



Appendix D 



RESPONSES TO THE 1967 SUMMER INSTITUTE PARTICIPANTS 
FOLLOW-UP QUESTIONNAIRE 



Three summer Institutes were held In Los Angeles, Philadelphia, 
and St. Louis In cooperation with Southwest Regional Laboratory, 
Research For Better Schools, c..id Central Midwestern Regional Educa- 
tional Laboratory. On December 4, 1967 all participants were mailed 
a follow-up questionnaire. The following are the results of the 
returned questionnaires. 





Los 

Angeles 


Phila- 

delphia 


St. 

Louis 


Total 


Percent 


Number of participants 


.n 


29 


26 


86 




Number of questionnaires 


returned to date 


23 


21 


14 


58 




Percentage of total ques- 


tlonnalres returned 


74% 


727. 


547. 


677. 




1. How manv hours a week 
were you engaged in 
research and/or evaluation 
activities last year? 


0 


3 


8 


3 


14 


24 


1 - 5 


13 


5 


9 


27 


47 


6 - 10 


1 


1 


l 


3 


5 


11 - 20 


3 


i 


0 


5 


9 


21 - 30 


1 


1 


1 


3 


5 


31 - 40 


2 


3 


0 


5 


9 


Omit 


0 


1 


0 


1 


2 




30 



Los Phi la- St. 

Angeles delphla Louis Total 



2. How many houvs a week are 
you currently engaged In 
research and/or evaluation 
activities? 



0 0 

1-5 7 

6-10 4 

11-20 4 

21-30 4 

31-40 4 

Omit 0 



3. What Is the nature of these 
activities ? 

Reviewing others' projects 



for funding purposes. 8 

Formulating research proj- 
ects which others carry 
out, 6 

Carrying out a preplanned 
project. 3 

Carrying out original 
research projects. 9 

Analysing data from a 
preplanned project. 3 

Supervising others' proj- 
ects. 4 

Conducting curriculum 
evaluation projects. 10 



1 1 2 

1 3 11 

3 8 15 

3 1 8 

0 1 5 

12 0 16 

1 0 1 



3 3 14 

4 4 14 

0 6 9 

6 1 16 

2 4 9 

4 2 10 

11 6 27 



O 

ERLC 



Percent 



3 

19 

26 

14 

9 

28 

2 

14 

14 

9 

16 

9 

10 

27 



31 



Los Phila- St. 

Angeles delphla Louis Tota l 



Write-In responses: 

Survey and Adminis- 
trative research 0 

Evaluating Title I 

ESEA 0 

Testing and test inter- 
pretation 0 

Omit 0 



4. Has the research proposal 
that you worked on during 
the Institute been com - 
pleted ? 



Yes 7 

No 16 

Omit 0 

5. Has it been funded ? 

Yes 7 

No 16 

Omit 0 

6. If to , by whom ? 

Government agency 3 

State agency 1 

Supervisor 0 

District 2 

Other: 

Phi Delta Kappa 1 

Rosenburg Foundation 1 

Omit 15 



2 0 2 

0 1 1 

0 1 1 

3 1 4 



2 5 14 

12 7 35 

7 2 9 

2 5 14 

12 7 35 

7 2 9 

0 3 6 

1 0 2 

0 1 1 

1 1 4 

0 0 1 

0 0 1 

19 9 43 




Percent 



2 

1 

1 

4 



24 

60 

16 

24 

60 

16 

10 

3 

2 

7 

2 

2 

74 



32 



Los Phila- St. 

Angeles delphla Louis Total 



7 . Check the highest stage of 

progress of vour prol ect . 

a. Discovering potential 

research problems 1 

b. Reviewing current 

practices and rele- 
vant research 3 

c. Preparing the research 

proposal 3 

d. Developing experimental 

materials 2 

e. Collecting data in the 

schools 6 

f. Analyzing data 3 

g. Preparing report and 

making recommendations 1 

h. Implementing changes 

consistent with recom- 
mendations 0 

i. Initiating follow-up 

study 0 

j. Preparing final report 0 

Write-in Responses: 

k. Proposal rejected as 

illegal 1 

Omit 3 



To what extent do you feel 




prepared to conduct this 




project? 




Well prepared 


10 


Moderately prepared 


11 



Inadequately prepared 



0 

2 



0 12 

0 3 6 

3 2 8 

4 17 

3 2 11 

1 2 6 

0 0 1 

0 0 0 

0 0 0 

0 0 0 

0 0 1 

10 3 16 



5 4 19 

9 8 28 

0 0 0 

7 2 11 



O 

ERiC 



Percent 



3 

10 

14 

12 

19 

10 

2 

0 

0 

0 

2 

28 

33 

48 

0 



Omit 



19 



33 



Los Phila- St. 

Angeles delohla Louis Total 



9 . How useful have you found the 
following Institute materials ? 



behavioral objectives 

frequently used 12 
occasionally used 9 
rarely or never used 0 
Omit 2 



13 

4 

2 

2 



5 30 

6 19 

2 4 

1 5 



experimental design 

frequently used 10 
occasionally used 9 
rarely or never used 2 
Omit 2 



5 2 17 

9 9 27 

2 2 6 

5 1 8 



types of studies and 
variables 

frequently used 6 

occasionally used 10 

rarely or never used 4 

Omit 3 

criterion measures 

frequently used 9 

occasionally used 8 

rarely or never used 3 

Omit 3 



7 4 17 

8 8 26 

4 19 

2 1 6 

5 2 16 

9 8 25 

4 3 10 

3 1 7 




Percent 



52 

33 

7 

9 

29 

47 

10 

14 

29 

45 

16 

10 

28 

43 

17 

12 



34 



L08 

Angeles 

instructional specification s 



frequently U8ed 5 
occasionally used 10 
rarely or never used 4 
Omit 4 



choosing statistical proce - 



dures 

frequently used 8 

occasionally used 8 

rarely or never used 5 

Oral t 2 

research report 

frequently used 9 

occasionally used 8 

rarely or never used 3 

Omit 3 

validity threats 

frequently used 7 

occasionally used 10 

rarely or never used 4 

Omit 2 



management of behaviora l 
consequences 



frequently used 6 
occasionally used 10 
rarely or never used 4 
Omit 3 



Phila- St. 
delphla Louis 




Total 



11 

22 

13 



10 



12 

21 

15 

10 



19 

23 

8 

8 



15 

29 

8 

6 



10 

26 

14 



8 




Percent 



19 
38 
26 
17 

20 
36 
26 
17 

33 

40 

14 

14 

26 

50 

14 

10 

17 

45 

24 

14 
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Los Phila- St. 

Angeles delphla Louis Total Percent 



research proposal 
frequently used 
ocasionally used 
rarely or never used 
Omit 

analysis of variance 
frequently used 
occasionally used 
rarely or never used 
Omit 



10 

7 

2 

4 

1 

9 

6 

7 



Questions 10 and 11 are at 
the end of this questionnaire 

12. How much are Institute 
materials assisting you 
with your activities ? 

very much 16 

some 7 

none at all 0 



Omit 



0 



13. What is your research 
budget for the current 
year ? 



100 - 500 
501 - 1000 
1000 - 5000 
above 5000 
Omit 



5 

0 

2 

4 

9 

3 



6 4 20 34 

7 7 21 36 

5 2 9 16 



3 



1 8 14 



10 2 3 

10 6 25 43 



5 6 17 29 

5 2 14 24 



14 

5 

0 

2 



7 

6 

0 

1 



37 

18 

0 

3 



64 

31 

0 

5 



8 

0 

0 

0 

7 

6 



3 

0 

2 

1 

4 
4 



16 

0 

4 

5 

20 

13 



28 

C 

7 

9 

34 

22 




36 



Los 

Angel es 



14, What was your research 
budget for the previous 
year ? 



0 10 

100 - 500 1 

501 - 1000 0 

1001 - 5000 3 

abovs 5000 5 



Omit 



15. Have you changed positions 
since you attended the 
Institute ? 

Yes 6 

No 17 

Omit 0 

16. Was your attendance at the 
Inst ] -vte instrumental to 
your changing positions ? 

Yes 4 

No 11 

Omit 8 



Phila- 

delphia 



8 

0 

0 

0 

7 

6 



7 

12 

2 



6 

5 

10 



St. 

Louis 



4 

1 

3 

0 

3 

3 



0 

14 

0 



0 

1 

13 



Total Percent 



22 38 

2 3 

3 5 

3 5 

15 26 

13 22 



13 22 

43 74 

2 3 



10 17 

17 29 

31 53 
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10. In dealing with your project have you come across activities that 
profitably could have greater emphasis in the Institute materials? 
If so, please describe below. 

Los Angeles: (Most often mentioned) 

1. Statistical Procedures 

2. More practical exercises 

3. Behavioral objectives 

4. How to sell a proposal 

5. Use of consultants in preparing research proposal 

Philadelphia: (Most often mentioned) 

1. Use of a computer programs 

2. Writing the Research Report 

3. Behavioral objectives 

4. Construction of survey and questionnaire instrument 

5. Statistical procedures 

6. Proposal preparation 

St. Louis: (Most often mentioned) 

1. Defining the research problem 

2. Writing the research report 

3. Analyzing variability 

4. Development of criterion measures 



0 
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11. Are there any areas in your present job for which you feel you 
could use some instructional materials, not presented in the 
Summer Institute? Please list areas in space below. 

Los Angeles: 

1. Task analysis procedures 

2. The use of computers in education 

3. Measuring objectives in the affective domain 

4. Evaluating behavioral objectives 

5. Product development procedures 
Philadelphia: 

1. Constructing questionnaires 

2. Pupil- teacher interaction and observation techniques 

3. Advanced design and statistics 

4. The use of computers in education 

5. Psychometric techniques 
St. Louis; 

1, Library research techniques 

2. Advanced design and statistics 

3, Constructing curriculum objectives 

4. Research technique for the classroom teacher 
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APPENDIX E 

Summer Research Institute Participants 
Philadelphia, Pennsylvania 



DELAWARE 


NEW YORK 


Jane B. Laskaris 
Research Assistant 
University of Delaware 


Robert P, Saunders 
Vice Principal 
Board of Education 
Middletown, New York 


FLORIDA 


OHIO 


Arthur E, Cohen 
Specialist, Research and 
Information 
Research Department 
Dade County Florida 


Melvin W. Herkner 
Administrative Director 
Personnel and Research 
Board of Education 


Ijourie S. Fisher 
Chairman, Department of 
Psychology and Education 
Dade Junior College 


PENNSYLVANIA 

John A. Cannon 
Teacher 


Frederic W. Lic r 
Systems Analyst 
Broward County Board of 
Public Instruction 


Archdiocese of Philadelphia 

Michelle M. Chaplin 
Teacher, West Philadelphia 
High School 
Board of Education 


NEW JERSEY 




Daniel G. Alexander 
High School Librarian 
Board of Education, Newark 


Jeffrey C. Douville 
Associate Curriculum Coordinator 
Bureau of Curriculum Development 
Department of Public Instruction 


George 0. Cureton 
Teacher 

Board of Education, Newark 


Pauline L. Edwards 
Coordinator of Research 
Abington School District 


Elizabeth Gerald 
Supervisor I 
Department of Education 
State of New Jersey 


Joseph M. Gavin 
Teacher 

Board of Education, Philadelphia 
Leonard E. Glassner 


Geraldine G. Sims 
Elementary Teacher 
Board of Education, Newark 


Research Associate 

Board of Education, Pittsburgh 



O 
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PENNSYLVAN IA (continued) 

Warren H. Groff 
Administrative Assistant 
to the Dean 
Temple University 

Ivan H. Guesman 

Assistant County Superintendent, 
Curriculum 

Department of Public Instruction 
Pennsylvania 

Margaret L. Havard 

Research Intern 

Research for Better Schools 

Charles W, Jones 

Education Evaluation Adviser 

Department of Public Instruction 

Gladys F, Jones 
Research Assistant 
Prekindergarten Program 
School District of Philadelphia 

Erma D, Keyes 
Counselor and Chairman, 

Guidance Department 
Downingtown Area School District 

Susan S, Klein 
Teacher 

School District of Philadelphia 

R. Lewis Rofman 

Supervisor -Research 

School District of Philadelphia 



Jay Smink 
Assistant Director 
Coordinating Unit for 
Vocational Education 
Bureau of Research 

Harold T. Smith 

Director, Secondary Curriculum 
School District City of Chester 

Alan H. Solomon 
Teacher 

School District of Philadelphia 

Elliott D. Waters 
Sixth Grade Teacher 
Board of Education 

Grace J. Weeks 

Supervisor in Research 

School District of Philadelphia 

Edward N. Whitney 

Evaluation Coordinator 

School District of Philadelphia 

George R. Young 
Teacher 

School District of Philadelphia 



VIRGINIA 

Kathryn J, Ripley 

Research Assistant 

Northern Virginia Community College 



O 
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Summer Research Institute Participants 
Los Angeles, California 



ARIZONA 

Douglas J, Adams 

Assistant Principal, junior high 
Tucson School District #1 

Lew S. Griffith 
General Curriculum Consultant 
Phoenix Elementary School 
District #1 

A, Frank Hansen 

Teacher and ETV Representative 

Douglas Public Schools 

William Poston, Jr, 
Administrative Assistant 
Research and Planning 
Mesa Public Schools 

Thomas L, Townzen 
Director of Elementary Education 
Scottsdale Elementary School 
District #48 

Sister Jean Ann Wilburn 
Teacher- Vice Principal 
St, Teresa School 



CALIFORNIA 

Marilyn Burns 

Consulting Counselor 

Los Angeles City School District 

Geraldine D. Ferguson 
Research and Evaluation Analyst 
Los Angeles City School District 
ESEA, Title III 

James A. Freda 
Vice Principal 
Buena Park School District 



John H. French 

Principal, Beverly Vista School 
Beverly Hills Unified School District 

Douglas E. Giles 
Principal, San Altos School 
Lemon Grove School District 

Raymond D, Hahn 
Teacher-Counselor 

Newport-Mesa Unified School District 

Otto A. Heinkel 

Research Assistant 

San Diego Junior Colleges 

Robert E. House 

Research Specialist 

Los Angeles City School District 

Leonard S. Kidd 

Teacher-Vice Principal 

San Diego City School District 

Charles F, Lee 
Superintendent 

Warner Union School District 
Rolf Lee 

Director of Research 

Federal Projects and Grants 

Simi Valley Unified School District 

Eugene F. McAdoo 

Research Specialist 

Los Angeles City School District 

James C, McDonald 

Superintendent- Principal 

Fallbrook Union High School District 

Rodney Mortenson 
Director, Pupil Personnel 
Oxnard School District 
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C ALIFORNIA (continued) 

John B, Nance 

Instructional Systems Consultant 
Fullerton Junior College 

Douglas L, Roscoe 

Research Assistant 

Riverside Unified School District 

Leonard Swenson 
Assistant Superintendent 
Instructional Services 
Valley Oaks Union School District 

Lester A, Tanner 
Principal (K-6) 

Newhall School District 

Jack R. Walker 
Coordinator of PACE-SIM 
ESEA, Title III 

San Bernardino County Superin- 
tendent of Schools 

Yvonne C. Watkins 

Director of Curriculum 

John F. Mudge, Superintendent 



Everett Waxtnan 

Director, Office of Research 
and Development 

Los Angeles City School District 
Grace L, Wiest 

Consultant, Compensatory Education 
Title I 

Riverside County Schools Office 



NEVADA 

David N. Anderson 
Director of Research 
Clark County School District 

Ronald L. McIntyre 

Coordinator, Media Selection System 

Clark County School District 

Sharon Y* Pearson 

Technical Writer 

Clark County School District 
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Summer Research Institute Participants 
St, Louis, Missouri 



ILLINOIS 

Newton L, Elliott 

Junior High Principal 

Alton Community Unit District #11 

Freeman Greer 
Administrative Assistant 
Cahokia School District #187 

Weldon Kendrick 
High School Superintendent 
Fairfield Community High School 
District #225 

Ruth D, Lahr 
Director, Title I, ESEA 
Granite City Community School 
District #9 

Theodore J. Nichols 
Supervisor or Testing and 
Research 

East St, Louis Public School 
District #189 

Donald C. Norwood 
Director of Statistics and 
Data Processing 
Office of Superintendent of 
Public Instruction 

James Tilashalski 
Principal, Elementary School 
Edwardsville Community Unit 
District #7 



KENTUCKY 

William W. Bolton 
Supervisor of Instruction 
Bourbon County Board of 
Education 



William B. Fisher, Jr. 

Principal of Adult Education and 
Teacher 

Jefferson County Board of Education 

Cletus L. Hubbs, Jr. 

High School Principal 
Hopkinsville Board of Education 

C. Hillman Mclntire 
Psychometrist 
Owensboro City Schools 

William E. Robinson 
Superintendent of Schools 
Carter County 

Elisabeth J. Tate 

Instructional Materials Supervisor 
Green County 



LOUISIANA 

James W. Gardner 
Assistant Director, ESEA 
Supervisor of Testing and Evaluation 
Acadia Parish School Board 



MISSOURI 

Russell R. Bastian 
Principal, Craig Elementary 
Parkway District 

Elmer Belsha 

Assistant Superintendent 
School District of Jennings 

Quincy C. Dickey 
Principal, Elementary School 
Berkeley School District 
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MISSOURI (continued) 

Brother Robert J. Godfrey 
Assistant Superintendent 
Catholic Archdiocese of St. Louis 

Walter D. Grigsby 

Curriculum Director 

St. Charles Public Schools 

Rerschel L. Neil 
Principal 

Parkway School District 
John H. Ross 

Assistant Superintendent of 
Schools 

Knox County School District R-I 



Sister Mary Noreen Corkery, R.S.M. 
Diocesan and Community Supervisor 
(Elementary) 

Superintendent of Catholic Schools 

Mansfield D. Neely, Sr. 

Regional Supervisor, Elementary 
Associate Director NEIP 
(Ford Foundation) 

James J. Webb 

Principal, Elementary 

Overton County Board of Education 

J. B. Whitman 
Superintendent of Schools 
Robert son County 



TENNESSEE 



Joe W. Clark 
Principal 

Humboldt City Board of Education 




